Handheld Device for Substitution from Vision to Audition

Sensorial substitution has great potential in rehabilitation, education, games, and in the creation of music and art. Current technologies allow us to develop sensorial substitution and sonification systems that would not have been imaginable two decades ago. It is desirable to let a large audience use and test sonification systems to provide feedback and improve their design. Handheld devices like smartphones or tablets include network connectivity (WIFI and/or Cellular radio) that can be used to transmit anonymous information about the configuration and strategies adopted by users. It is now feasible to obtain feedback from any user of substitution and sonification technology and not only from a limited number of subjects in the laboratory. Testing in the field with a large number of users is now possible thanks to telecommunication networks and machine learning tools to analyze big data. This work presents a handheld implementation of a simple video sonification system designed to test the acceptability of vision to audition substitution systems and in the near future to provide feedback from users. A first beta version was publicly released in November 2013 as an iOS application for large scale testing. The extended abstract introduces the interface and the underlying technology.